Planetary turbulent flows are observed to self-organize into large-scale structures such as zonal jets and coherent vortices. One of the simplest models of planetary turbulence is obtained by considering a barotropic flow on a beta-plane channel with turbulence sustained by random stirring. Nonlinear integrations of this model show that as the energy input rate of the forcing is increased, the homogeneity of the flow is broken with the emergence of non-zonal, coherent, westward propagating structures and at larger energy input rates by the emergence of zonal jets. We study the emergence of non-zonal coherent structures using a non-equilibrium statistical theory, stochastic structural stability theory (S3T, previously referred to as SSST). S3T directly models a second-order approximation to the statistical mean turbulent state and allows the identification of statistical turbulent equilibria and study of their stability. Using S3T, the bifurcation properties of the homogeneous state in barotropic beta-plane turbulence are determined. Analytic expressions for the zonal and non-zonal large-scale coherent flows that emerge as a result of structural instability are obtained. Through numerical integrations of the S3T dynamical system, it is found that the unstable structures equilibrate at finite amplitude. Numerical simulations of the nonlinear equations confirm the characteristics (scale, amplitude and phase speed) of the structures predicted by S3T.